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Abstract 

Background: Funding agencies constitute one essential pillar for policy makers, researchers and health service 
delivery institutions. Such agencies are increasingly providing support for science implementation. In this paper, we 
investigate health research funding agencies and how they support the integration of science into policy, and of 
science into practice, and vice versa. 

Methods: We selected six countries: Australia, The Netherlands, France, Canada, England and the United States. For 
13 funding agencies, we compared their intentions to support, their actions related to science integration into 
policy and practice, and the reported benefits of this integration. We did a qualitative content analysis of the 
reports and information provided on the funding agencies' websites. 

Results: Most funding agencies emphasized the importance of science integration into policy and practice in their 
strategic orientation, and stated how this integration was structured. Their funding activities were embedded in the 
push, pull, or linkage/exchange knowledge transfer model. However, few program funding efforts were based on 
all three models. The agencies reported more often on the benefits of integration on practice, rather than on 
policy. External programs that were funded largely covered science integration into policy and practice at the end 
of grant stage, while overlooking the initial stages. Finally, external funding actions were more prominent than 
internally initiated bridging activities and training activities on such integration. 

Conclusions: This paper contributes to research on science implementation because it goes beyond the two 
community model of researchers versus end users, to include funding agencies. Users of knowledge may be end 
users in health organizations like hospitals; civil servants assigned to decision making positions within funding 
agencies; civil servants outside of the Ministry of Health, such as the Ministry of the Environment; politicians 
deciding on health-related legislation; or even university researchers whose work builds on previous research. This 
heterogeneous sample of users may require different user-specific mechanisms for research initiation, development 
and dissemination. This paper builds the foundation for further discussion on science implementation from the 
perspective of funding agencies in the health field. In general, case studies can help in identifying best practices for 
evidence-informed decision making. 

Keywords: Knowledge management system, Knowledge funding agencies, Knowledge use/utilization, International, 
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Background 

A key challenge facing health services research and sci- 
ence policy is understanding and optimizing the integra- 
tion of science into both policy and the practices of 
professionals. This paper investigates how health re- 
search funding agencies support the integration of sci- 
ence into policy and practice. 

First, there is confusion and misunderstanding about 
terms such as knowledge transfer, knowledge translation, 
implementation and research utilization, and even more 
so, about such concepts as 'moving knowledge into action' 
[1]. This has led to calls for an improved understanding of 
the uses of research [2]. Only a few fields of research — 
notably health service research [3-9] and knowledge trans- 
fer research [10,11] —have investigated science integration 
into policy and practice (SIPP). In fact, health services re- 
search, especially research concerned with the use and im- 
plementation of research results in policy, has become so 
intertwined with the field of knowledge transfer that calls 
for efforts have been made to compare and clarify terms 
used in the two fields. 

Knowledge transfer spans a number of different fields, 
including communication with the public, communica- 
tion with media, and collaborative approaches to science. 
In this paper, the main focus is the direct integration of 
science into decision making, both in policy and prac- 
tice. We do not examine the indirect influences of sci- 
ence on policy making, such as lobbying activities and 
public opinion. Rather, we focus on evidence-informed 
decision making aimed at modifying policy and practice, 
such as SIPP and methods prior to SIPP in the process 
[12]. The line separating knowledge transfer and know- 
ledge translation is unclear, with some authors advocat- 
ing that concepts like technology transfer, continuing 
education and commercialization not be included under 
these terms [13]. We will not enter into this debate here; 
rather, we will focus on the importance of research- 
based knowledge in the decision-making process at the 
level of both policy and practice. 

In the 1970s, concern over knowledge-based decision 
making increased among researchers. Research utilization 
[14] encompasses conceptual use, instrumental use and 
strategic purposeful use. Organizations — funding agencies 
in particular — have been interested in licensing, acquiring, 
and commercializing biomedical discoveries for years. Re- 
cently, stakeholders have championed the integration of 
health-related sciences, human sciences, and biomedical 
sciences into policy and practice. Dedicated funding, the 
identification of strategic objectives, and the implementa- 
tion of internal activities within funding agencies all foster 
SIPP. In the literature on science policy, or how the pol- 
icies of national governments, supranational organiza- 
tions, public science funding organizations, and large 
public research organizations aimed at influencing the 



production of scientific knowledge [15],' funding agencies 
have been analyzed [12,16,17]. 

How science policies for SIPP are ingrained in the re- 
search agenda at the national level [18-20], how they are 
adjusted to the educational sphere [21], how they are 
formatted under external pressures [22], and how they 
are included in citizen and democratic procedures are all 
important issues. The dialogue around, and actions 
aimed at, producing stronger investigative SIPP research 
highlights two elements: the need to consider other key 
players beyond just scientists and policy makers, and the 
benefits of understanding the influence of scrutinizing 
institutions on the role of science in policy making [23]. 

This paper identifies funding agencies as a key player and 
contributor to SIPP. This premise means we need to study 
funding agencies more closely. Not only do we need to see 
them as a key player, as already suggested by Sutherland 
[23], we need to study their role as a central element, rather 
than just one variable in the overall dynamics of SIPP. 

This paper is innovative in several ways: it focuses on 
funding agencies as a key player in SIPP rather than just 
a component of the outside environment as previous au- 
thors have done [12,16,17]; it compares funding agencies 
on SIPP and examines the policy and practice achieve- 
ments of program funding; it expands the traditional 
conceptualization that knowledge transfer is centered 
around practitioners, policy makers, and academics; it 
adds to the growing trend for research focused on policy 
integration in the broader framework of health service 
research; and it focuses on 'the external social structures 
and processes that influence individual knowing and the 
production of collective knowledge' [24]. 

In Western countries, the agendas of funding agencies 
prioritize national research and orchestrate research 
funds to answer current and emerging health challenges 
that have the potential to improve the health of their 
populations. Funding agencies constitute operational pil- 
lars for policy makers, researchers, and service delivery 
institutions like hospitals and health departments. This 
paper investigates the role of funding agencies in SIPP. 

We provide an overview of the knowledge translation 
mechanisms of funding agencies as it relates to the 
health sector. There is a need to identify and discuss the 
challenges to integrating research into policy and into 
practice in a timely manner, and to enhance the integra- 
tion of research findings into policy — at the macro, local 
and in practices at the micro levels. 

To date, a few empirical studies have provided details 
about SIPP funding: most notably, a study that com- 
pared European health services research at the national 
level [5], and another that compared health research 
funding at the agency level [25]. Both studies relied on a 
small number of informants. In the first study, 34 coun- 
tries were surveyed in 2007 to 2011, while in the second, 
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members from 33 agencies were interviewed in 2003 
and 2004. Both studies presented results from an in- 
depth analysis of official funding agency documents. The 
European comparisons mainly looked at both public and 
private funds, policy making, and the activities being 
promoted. In our study, we examine these aspects, but 
also add the documented benefits of funding, both in 
and outside of the policy domain. We include 'internal' 
SIPP orientation strategies by funding agencies, along 
with agency activities that support the innovation cycle 
for beneficiaries of funds. We present a comparative, 
international analysis of the funding agencies' require- 
ments, funding strategies, and the types of activities they 
fund. Therefore, this paper continues the international 
focus, while also examining the role that funding agen- 
cies play in SIPP. It also contributes to the literature on 
SIPP by emphasizing health research funding agencies 
and inward strategic positioning, all while serving to 
bring the current literature up to date. 

The overall objective of this paper is to determine 
what are the intentions, actions, and benefits of the inte- 
gration of science into policy and practice in health- 
related funding agencies throughout the world. Our 
comparisons will help answer questions like: What kind 
of visibility do funding agencies give to SIPP? How do 
funding agencies promote SIPP? What impacts do fund- 
ing agencies have on practice and policy? 

Methods 

Selection of funding agencies 

We selected funding agencies from six countries: Australia, 
The Netherlands, France, Canada, England, and the 
Unites States. The agency had to be national or supra- 
national in scope, fund health-related research, and a 
major provider of research funds. We questioned pro- 
fessors, researchers, members of funding agencies, 
managers and health service providers who were mobi- 
lized to identify the characteristics of the major provider 
of research funds category, identifying the 'contrast or 
continuum on knowledge translation engagement' [25] . 
Granting councils are: 

Australia 

National Health and Medical Research Council 
(NHMRC) 

Australian Research Council (ARC) 
The Netherlands 

The Netherlands Organization for Health Research and 
Development (ZonMW) 

France 

Institut National de la Sante et de la Recherche 
Medicale (INSERM) 



Centre National de la Recherche Scientifique (CNRS) 
Agence Nationale de la Recherche (ANR) 

Canada 

Canadian Institutes of Health Research (CIHR) 
Natural Sciences and Engineering Research Council 
(NSERC) 

Social Sciences and Humanities Research Council of 
Canada (SSHRC) 

England 

National Institute for Health Research (NIHR) 
USA 

Food and Drug Administration (FDA) 
Environmental Protection Agency (EPA) 
National Institutes of Health (NIH) 

Data sources 

We selected published reports and information provided 
on the websites of the selected funding agencies (see 
Additional file 1). We extracted information from stra- 
tegic plans, mission statements, organizational charts, 
reports on available funding for research and internal 
productions, success reports, and descriptions of agency 
activities. Given that the interest in SIPP is fairly recent, 
we restricted our search of key resources to the period 
of 2005 to 2011. 

The empirical secondary data that we gathered from 
these sources was limited: these data do not necessarily re- 
flect the most recent and emerging trends in the policies 
and decisions being made by funding agencies. To make up 
for this limitation, we searched for the latest available and 
accessible strategic plans and organizational documents. 
Such an approach may have introduced a positive bias and 
lead to a more selective discourse disproportionately cen- 
tered around ongoing activities and the results obtained for 
these particular activities. However, we believe that this bias 
is less serious because we are interested in achievements ra- 
ther than failures and barriers. We included reports on the 
impacts of funding on policy or practices when the data 
were detailed and relevant to the grant received. While 
source data analysis may not have allowed us to compile in- 
formation on all of the internal dynamics of funding, such 
dynamics were not our main point of interest. 

Using secondary data has the advantages of providing 
a level of depth similar from agency to agency, and 
sometimes these secondary data have similar categories, 
thereby making the comparisons more accurate and 
straightforward. The reports and information published 
on websites are produced after a review process and ap- 
proval by the department responsible for editing and 
publishing official data, thereby ensuring a consistent, 
shared view on topics of interest and, ultimately a 



Smits and Denis Implementation Science 2014, 9:28 
http://www.implementationscience.eom/content/9/1/28 



Page 4 of 12 



comprehensive data set. It would have been difficult to 
obtain such a large and diverse dataset any other way: a 
single representative from one institution simply could 
not have provided such variety in content. We believe 
reports and official website information are a useful 
dataset for studying SIPP in funding agencies at this 
point in the development of research about science inte- 
gration into policy and health. 

Note that our data collection relied on information 
available on the agencies' websites. The uniformity of 
the data published by a particular agency cannot be 
guaranteed by our methodology. There is reason to be- 
lieve the selected agencies publish data that are not so 
different from one another. Indeed, the websites reveal 
each agency's formal representation of its own activ- 
ities. Moreover, the information published by agencies 
on their websites is directed at the public, researchers, 
policy makers, and practitioners. It is their public 
portrait. 

Our analysis of the websites generated secondary, ter- 
tiary, and endless snowball references as a result of navi- 
gating the agencies' websites plus those of linked 
agencies and ministries. We pulled documents that were 
directly posted on the agencies' own websites. Relevant 
information was either easily accessible via the home- 
page, or it was less available, being stored in archives or 
identifiable by the title on webpages. We browsed the 
whole website, including archives, paying particular at- 
tention to webpages related to strategy and funding. Of 
course, a key informant in each agency could have vali- 
dated the documents we retrieved, but we sought only 
to examine documentation. 

Identification of variables 

We investigated three main variables of SIPP and their 
corresponding dimensions (Table 1): 

1. Intentions: Health research funding agencies set 
health as a priority and then devise ways to support 
and foster the integration of science into policy and 
practice. We examined the following dimensions of 
intentions: organizational visibility of SIPP (absent, 
external, internal) and the purpose of SIPP 
(management, network, capacity building). 

2. Actions: Health research funding agencies organize 
both internal and external SIPP-related activities. 
Actions are any specific activities by funding 
agencies to ensure the materialization of science into 
policy and practice, and to encourage evidence- 
based or evidence-informed policy or practices. We 
examined the following dimensions of actions: type 
of external actions (push, pull, linkage and exchange 
(L&E) funding), types of internal actions (reinforcing 
capability, process, codification), and steps in the 



Table 1 Summary of variables 


Variable 


Dimensions 


Intention 


Organizational visibility 




Purpose of SIPP 


Action 


Type of external actions 




Type of internal actions 




Steps of innovation 


Benefits 


Type of influences 




Specific influences on policy 




Specific influences on practice 




Specific influences on service delivery 



Policy refers to orientations taken by funding agencies on knowledge transfer. 
Practice refers to the operational translation of programs and activities. 



innovation process (synthesis, initiation, 
development, end of grant, science, training). 
3. Benefits: Funding agencies collect evidence of the 
empirical benefits and return on investment for their 
funding activities. Benefits are the consequences — 
often positive outputs and outcomes — that 
contribute to, or are generated by, specifically 
targeted SIPP activities. We examined the following 
dimensions of benefits: type of influences (policy, 
practice, service), specific influences on policy (local, 
national, international), specific influences on 
practice (improvement, new, modification of 
regulations), and specific influences on service 
delivery (financial, human, material). 

Results 

The agencies used a variety of terms and concepts when 
talking about SIPP in their strategic and mission state- 
ments: testing knowledge, applying knowledge (applica- 
tion), transfer, translate research/knowledge, valorization, 
knowledge management, put to real-world use, integrate 
science and technology, science coordination with prior- 
ities, informed policy advice, use of knowledge/utilization 
of science, strategic management of licenses, innovation, 
mobilizing knowledge, commercialization, evidence-based 
approach, and consider results in the decision making 
process. 

Analysis of intentions 
Organizational visibility 

The importance and readiness of SIPP for funding agencies 
is partially reflected in their organizational charts, mission 
statements, and strategic objectives. The organizational 
charts sometimes attribute the responsibility of SIPP to dis- 
tinct directorates, departments and/or branches (Table 2). 
For example, SSHRC has a separate division for knowledge 
mobilization and program integration, unlike ARC, which 
does not have any specific divisions. Most of the funding 
agencies manage their SIPP mission from within; usually it 
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Table 2 Organizational visibility of SIPP in funding agencies 


Visibility 


Funding agencies 


Absent 


ARC no specific directions 


Externa 


INSERM. INSERM Transfert SA a private entity from INSERM 


nternal 


CNRS Direction of innovation and relationships with companies (translation) 




ANR. Partnership and competitiveness; investment in the future (translation) 




EPA Office of science policy 




NIH Office of science policy analysis 




CIHR Knowledge translation branch; partnerships and citizen engagement branch; communications and public outreach branch 




SSHRC Knowledge mobilization and program integration division 




NHMRC Research investment direction/Prog Grants, Knowledge Translation & Capacity Building 




NSERC Research partnerships program directorate; research grants and scholarship directorate 



For ZonMw and NIHR, we found no data on organizational structure for SIPP. Please note that NIHR does commission research on SIPP through the NHS and the 
Health Services and Delivery Research Programme [35]. 



is under the responsibility of one of their directorates 
or branches. For example, SSHRC has a knowledge 
mobilization and program integration division, while 
NHMRC has research investment direction a /Know- 
ledge Translation & Capacity Building. In France, 
INSERM manages some of its SIPP objectives through 
a private entity: INSERM Transfer SA. This group is in 
charge of putting highly promising biomedical innova- 
tions sponsored by INSERM funds into action; it diag- 
noses, accompanies, helps manage, protects, and 
supports innovations on the way to commercialization. 
One could say that INSERM is involved in translational 
research rather than knowledge transfer per se, al- 
though there needs to be some adoption of knowledge 
with partner companies and so this activity qualifies as 
knowledge transfer in our analysis. 

Purpose of SIPP 

Strategic support for SIPP was embedded in various 
strategies devoted to management, exchange, and/or 
capacity building. SIPP could be narrowed down to man- 
agement objectives, in which knowledge informs the de- 
velopment or improvement of new practices and getting 
a product out on the market. In such cases, the funding 
agencies put an operational emphasis on SIPP. For ex- 
ample, INSERM placed an emphasis on managing the 
transfer of technology: 

'Its principal mission is to coordinate the valorization 
of medical innovations from research laboratories', 
[our translation] 

Organizational chart: INSERM Transfert SA a private 
entity from INSERM 

Source: translation. INSERM Transfert mission. 
Website. 



Some of the funding agencies had a broader scope and 
emphasized certain processes involved in SIPP. They 
favored a flow of information through contacts and ex- 
changes between researchers and policy makers. For ex- 
ample, CIHR referred to knowledge translation and 
insisted on the importance of dynamic and iterative ex- 
changes between the two communities: 

'Knowledge translation (KT) is defined as a dynamic 
and iterative process [...] to improve the health of 
Canadians, provide more effective health services and 
products and strengthen the health care system. This 
process takes place within a complex system of 
interactions between researchers and knowledge users 
which may vary in intensity, complexity and level of 
engagement depending on the nature of the research 
and the findings, as well as the needs of the particular 
knowledge user'. 

Organizational chart: Knowledge translation branch; 
partnerships and citizen engagement branch; 
communications and public outreach branch 
Source: CIHR. DPR 2009-2010. 

SIPP may build on the influence it has on policy and 
policy makers to increase its influential capacity in the 
legislative branch. In Australia, both funding agencies 
we studied sought to contribute to individual and 
organizational learning: NHMRC by translating know- 
ledge to a broad audience in order to improve policy 
and practice, and ARC by playing an advisory role in 
policy making. 

'Policy: To provide informed high quality policy ad- 
vice to Government — through participation in policy 
forums and Government reviews, informed stakeholder 
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consultations, evaluation and ongoing monitoring of 
performance'. 

Organizational chart: No specific directions. 
Source: ARC. 2010. Strategic Plan 2010-11 to 
2012-13. 

To summarize, our comparison of the intentions of 
funding agencies shows that: 

1. Most funding agencies emphasized SIPP through 
specific organizational structures; 

2. Science was integrated into policy and practice 
through various operational means and by direct 
support: through improvements to the management 
of SIPP, encouraging the flow of information 
between actors in SIPP, and improving learning 
related to knowledge; 

3. The internal structure of the funding agencies, 
combined with relevant funds, increased the 
likelihood of SIPP. 

Analysis of actions 

Types of external actions (pull, push and L&E models) 

Informed decision making was often conceptualized in 
terms of how to encourage mechanisms that would 
bring science into the decision-making arena — that is in 
a push fashion. The push-pull model of knowledge 
translation is famous for distinguishing between mecha- 
nisms driven by science (push) and those driven by the 
demands of practitioners or policy makers (pull). These 
models consider two communities in interaction: the sci- 
entific community of researchers and the practice com- 
munity of decision-makers, politicians, local community 
members, et al. A third model advocates for consider- 
ation of interactions that take place along the innovation 
cycle and make science useful [26]. Such L&E [10], or 
deliberate models, rely on the co-construction of applied 
knowledge [27] and the relevance of applied research to 
both users and researchers. During research develop- 
ment, data collection and interpretation tasks can be 
shared. Most of the agencies we examined funded pro- 
grams based on one or two of these models, either push, 
pull, or L&E. Few provided funding based on all three 
types of SIPP models, with the exception of NHMRC, 
ZonMW, NSERC and NIHR (Table 3, line 1). 

ZonMW's Ethnic Minorities and Health Care Program 
belongs to the push model driven by science: 

'The Ethnic Minorities and Health Care Programme 
aims to promote the implementation of knowledge and 
skills available in this area. It targets both providers and 
users of care. The programme focuses on improving 
somatic curative care, an area in which knowledge 



and methods are available that could potentially 
improve care for ethnic minorities in the 
Netherlands. The aim is to spread this knowledge 
and these methods, and ensure they become part 
of mainstream care'. 

NHMRC's urgent research program reflects a pull 
model driven by policy needs: 

'Research that must be undertaken rapidly in response 
to a threat to public health. The threat may be 
generalised, or specific to a particular group of 
individuals, and may be identified as either a current 
major problem, a potential major problem or a 
problem that is expected to increase in the future. 
The main catalysts for urgent research will be the fact 
that a disease or illness, or its variant(s), is previously 
unknown or unidentified, and has a high morbidity 
and/or mortality rate, thus garners media coverage 
and public and/or governmental concern. This 
definition covers the range of possibilities at the 
cellular level (the identification of the disease or 
illness, and its variants), to those at the public and 
population level, and then the level of risk to a nation. 
For example, if a disease poses a probable or actual 
threat to the national and/ or local economy by 
hindering exports, tourism, agriculture, and so on. 
The SRDC has outlined a process for considering 
requests for urgent research based on this decision'. 

ANR's competitive pole funding program belongs to 
the linkage and exchange model: 

A competitive pole is constituted by companies, 
research laboratories and training centers in a 
geographic area in order to favor cooperation and 
exchange', [our translation] 

In the agencies studied, the creation and development 
of research in collaboration with users (L&E model) took 
the form of meetings to explore prospective subjects of 
interest, on-site training opportunities for practitioners to 
come to research settings, and inviting researchers to join 
decision making settings. The NIHR in England provided 
financial support for partnerships with health service pro- 
viders through its Service Delivery and Organization 
(SDO), where research results could be presented to prac- 
titioners and practitioners could present their needs. 
AHSC Academic Health Science Centers have brought to- 
gether research units, health care services, and education 
institutions, thus promoting cross-fertilization between re- 
search and practice. 

Funding based on the push model was for the transfer of 
existing research to users via publications in peer-reviewed 
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Table 3 Funding agencies program funds 



Line 1. Program funds in a push/pull/linkage and exchange (L&E) model 


Push 




Pull L&E 


3 modes 


% of funding 69 
agencies 




62 62 


23 


Line 2. Eligible expenses per innovation cycle steps 


Synthesis Initiation Development 




End of grant Science Training 


All steps 


% of funding 38 54 69 
agencies 




77 31 62 


23 


Line 3. Internal activities related to SIPP 


Capabilities Process 




Codification 


All 3 


% of funding 85 54 
agencies 




46 


46 


Line 4. Benefits of funded projects 


Policy Practice 


Service delivery 


3 types 


Local National International Improvement New 

of practices 


Modification of Financial Human Material 
practice regulations 


Policy, practice and 
service delivery 


% of funding 15 8 8 31 31 




8 8 8 23 


8 


agencies 15 45 




23 


8 



journals or practitioner-oriented journals, or through face-to- 
face dissemination in forums and conferences. For example, 
the EPA established a Clean Diesel Emerging Technologies 
Program to help companies purchase innovative technolo- 
gies to control fleet-related diesel emissions. Technologies 
existed. The aim was to make innovative, targeted technolo- 
gies available and financially accessible to companies. 

In funding based on the pull model, research may be 
influenced by decision makers, either by focusing on pri- 
orities, emergencies, threats, or an urgent need for data. 
For example, an infectious outbreak requires rapid 
knowledge and treatment solution, which requires that 
research priorities become oriented by decision makers. 
Programs that targeted science included the develop- 
ment of influenza vaccines (the NIVAREC center funded 
by ZonMw), the development of specific, applied know- 
ledge (e.g., ZonMw's Chronic Fatigue Syndrome Pro- 
gram, which encompasses the science of chronic fatigue 
syndrome, treatment and rehabilitation methods), pro- 
grams based on priority areas such as Aboriginal re- 
search (SSHERC), and strategic awards (NHMRC). 

The funding agencies that we studied all funded either 
push, pull, or L&E research (Table 3, line 1). Many agen- 
cies funded initiatives based on at least two models 
(push and pull), but these projects represented less than 
one-third of all documented funding (all models). 

Types of internal actions 

Internal activities carried out by the agencies themselves 
help develop organizational knowledge and SIPP. The in- 
ternal activities that we documented varied in nature. 
Some activities reinforced capabilities, such as classes and 
SIPP training sessions, or helped build organizational 



knowledge (capability). Some activities were more dy- 
namic and helpful in building organizational knowledge, 
such as creating knowledge brokers, forums, and commu- 
nities of practice. They put actors into situations where 
they could discuss and/or solve problems or become more 
comfortable with specific topics or issues; in short, they 
helped build organizational knowledge understood as a 
process. Other activities favored the provision of opera- 
tionalized information through guides and handbooks. 
These helped the organization build knowledge understood 
as a codification. Capability, process, and codification are 
ways to categorize knowledge [28], and an organization can 
use any of these as a basis of its actions. 

Activities that reinforce capabilities 

Most funding agencies offered services that facilitated the 
use of evidence-based knowledge in decision making. 
Some resources, like online portals, assisted in identifying 
and tracking funding opportunities (NHMRC). Three-day 
courses or a center for posting reviews allowed researchers 
to learn and share experiences (NIHR) and diffuse success 
stories, to share ideas and good practices with other grant 
applicants and practitioners (ARC, NSERC, CIHR and 
INSERM). Publishing lists of licensed innovations created 
opportunities for companies to carry out discovery research 
(INSERM). And classes, summer schools (CIHR), and lec- 
ture sessions for researchers, policy makers, or parliament 
members (CIHR, AR) served to train actors for SIPP. 

Activities that reinforce processes 

Our documentation shows that not only resources, but 
also active mechanisms favored SIPP. These mechanisms 
put the utilization of science in the decision-making 
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process in a dynamic context that enabled researchers 
and decision makers to interact through networks of 
trainees (CIHR), encouraged journalistic interest in science 
discoveries through journalism awards (CIHR), enabled 
companies to consult groups of research expertise 
(INSERM), and allowed professionals to interact through 
communities of practices (NHMRC). EPA had policy agree- 
ments that allowed for formal liaisoning and the communi- 
cation of scientific data to decision makers and official 
regional coordinators for research and development. 

Activities that reinforce codification 

Making formalized knowledge useable relies on inform- 
ative and descriptive documents. Summaries of research 
results can be translated into useful tools for practitioners, 
like the targeted summary of guidelines by NHMRC. 
Among informative documents, we also found knowledge 
transfer handbooks (CIHR). To translate research findings, 
the funding agencies also developed more prescriptive 
tools, such as targeted summaries of guidelines (NHMRC) 
or specific guidelines (NIH, USA, EPA, USA), and even 
rules (EPA, USA), agreements, and policies (NIHR). 

In summary, we found that the agencies and ministries 
developed a variety of activities, especially capabilities, to 
make knowledge useful and useable (Table 3, line 3). 
These activities enriched the organizations and enabled 
them to use the science and knowledge produced. 

Steps of innovation 

Funding for eligible expenses through funded programs, 
which may be based on a push, pull, or L&E model, may 
cover a number of different steps in the innovation cycle 
[12]. The first step in innovation is initiation, in which 
connections between decision makers and researchers are 
established and topics of interest fine-tuned. The second 
step is development, in which the research is conducted 
and then data collected and analyzed. Then there is syn- 
thesis (end of grant), which involves the presentation of 
research results to the public through conferences and 
written diffusion methods like reports. Sometimes reports 
synthesize previous results in the field. Research efforts 
may center on either generic research topics, priority 
areas, or SIPP itself. An eligible applicant, such as a fellow- 
ship candidate, institution or group of actors, may, for ex- 
ample, benefit from classes on SIPP or even do a 
temporary stay at a federal institution in order to get a dir- 
ect feel of how and what is done inside public administra- 
tion (CIHR). The goal of such training is to help the 
applicant translate research into polity and practices. 

The most common steps that were funded in the 
agencies we studied were end of grant, research devel- 
opment, and training in SIPP (Table 3, line 2). Only 
two agencies funded all steps in the innovation cycle. 
While development and end-of-grant projects appeared 



to be more frequently funded than knowledge synthesis 
or SIPP training projects, the funding agencies did 
sometimes have internal SIPP-related activities that 
were not related to funding competitions. 

We may draw the following conclusions about the 
funding agencies' involvement in the innovation process: 

1. The push, pull and L&E models of knowledge 
transfer were each supported by a slight majority of 
funding agencies 

2. Few of the funding agencies relied on all three 
models, meaning that all of the complementary 
dynamics of the knowledge processes were not 
always covered in their funding efforts; 

3. Externally funded programs largely covered SIPP at 
the end-of-grant stage of the innovation process, 
promoting diffusion in reviews, conferences, 
etcetera; 

4. Internally developed SIPP activities mainly focused 
on providing technical support, such as lectures via 
an online portal, and such activities were less 
frequent than those aimed at bridging and 
translating knowledge. 

Analysis of benefits 

Tracking health service research utilization is not an easy 
task [3,29]. Here, we report on SIPP-related success stor- 
ies, providing examples, individual cases, and positive re- 
sults in which funded activities have led to changes in the 
use of resources for service delivery, changes in the pro- 
duction of service delivery, or in policy making. 

Types of benefits 

The funding agencies mentioned some benefits for prac- 
tice and policy of their SIPP-related support. We did not 
find any references to the impact of SIPP-related internal 
activities. The granting agencies reported the conse- 
quences of SIPP under headings like 'impact spotlights,' 
'impact stories,' 'outcomes,' and 'results.' The benefits 
discussed in published reports took two forms: direct re- 
sults of the funded research, such as a decrease in the 
level of cholesterol in the intervention group, and the 
impact of the funded research beyond the immediate 
scientific community, such as an influence on decision 
making in the form of informing guidelines used to de- 
velop a new policy. NHMRC's Evaluation and Outcomes 
Working Committee defines direct results as the know- 
ledge creation outcome of research [30]. 

For the purposes of this research, we are interested in 
the indirect outcomes and influences of SIPP benefits. 
These benefits encompass both macro and micro issues. 
They emerge at the policy level, at the practice/ 
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production level, and at the level of service delivery to 
clients and the population (Table 3, line 4). 

Benefits at the policy level 

The modifications of policies by SIPP funding could have 
local, national, or international effects (Table 3, line 4). 
When the modifications were local, they affected the prac- 
tices of groups or organizations. Sometimes national pol- 
icies were modified through adjustments to regional or 
federal policies. The third type of SIPP influence 
was international, whereby the policies of international 
institutions or non-governmental organizations made 
changes to their policies based on practices elsewhere in 
the world. 

Benefits at the practice level 

Some of the SIPP funds supported operational modifica- 
tions aimed at improving processes and procedures. 
These mechanisms were sometimes implemented at the 
country level (Table 3, line 4). When regulations chan- 
ged under the impetus of research data, the result could 
be the launch of a brand new process or the alteration 
of existing ones. Alterations could be accomplished 
through the deletion of outdated information that re- 
quires modifications, or though the addition of informa- 
tion to improve current practices. The modification or 
introduction of a new practice or regulation on practices 
should, theoretically, lead to improved service delivery. 
For engineering projects that lead to conclusive out- 
comes, technologies could be used by other industries, 
thereby allowing for the creation of new products, diag- 
nostic tools or production lines. 

Benefits at the service delivery level 

Eventually, research outcomes served as a basis for the 
adjustment of services delivered to the population — to 
patients, to clients, and to other countries (Table 3, line 
4). Service delivery requires human resources, material 
resources and financial investment. We documented ad- 
justment in all of these types of resources based on evi- 
dence gathered in research. 

We traced the reported success stories to a number of 
different fields: policy making, practice organization, and 
service delivery (Table 3, line 4). A common impact of 
the funding agencies funds was the adjustment of prac- 
tical rather than policy aspects, and science delivery. 
The funding agencies, especially those focused on med- 
ical development and technology, often reported the 
commercialization of research ideas — for example, the 
development of vaccines and medical technologies. 
While success stories related to SIPP were often re- 
ported under sections pertaining to the immediate re- 
sults of research, the influence of research on service 
delivery, practice, and policy making may not be 



consistently gathered through funded projects nor re- 
ported in funding agencies like CNRS. 

To summarize, our analysis of funding agencies re- 
vealed the following: 

1. There is evidence of benefits at the policy, practice 
and service delivery levels; 

2. There is evidence that CIHR publicly documents 
SIPP benefits in a more consistent way than do the 
other funding agencies studied; 

3. There is evidence that the benefits of SIPP on 
practice are more common than reported. 

Discussion and conclusions 

Conclusions 

The goal of this paper was to address the following 
questions: what visibility do funding agencies give to 
SIPP? How do funding agencies promote SIPP? What 
impacts do funding agencies have on practice and pol- 
icy? It is difficult to draw a complete and accurate por- 
trait of SIPP due to the diversity of terms being used to 
designate knowledge transfer and evidence-informed de- 
cision making for policy and practice. This no doubt re- 
flects the fact that this is still a field under construction. 
Our findings support those of Tetroe et al. [25], who 
found that funding agencies from a variety of countries 
defined 'knowledge transfer' in many different ways [25]. 
Their results were based on interviews with health re- 
search funding agencies, ministries, non-governmental 
organizations, researchers, and on an analysis of the 
funding and internal activities of the agencies. 

Apart from the variety of models, definitions, and 
measures that agencies use, there is a need to clarify in- 
dividual and organizational brokering interventions [31]; 
reduce inconsistency in publication dissemination meas- 
urement and guidance [32]; and adapt health research 
systems to policy makers and to stakeholders' policy- 
making systems [5,33]. We need to take into consider- 
ation issues related to researcher autonomy (knowledge, 
career based on productivity, employment stability); pol- 
icy motivation (agendas, urgent needs, career based on 
publicity, re-election); and the closeness of funding 
agencies to researchers and policy makers. Based on our 
results, we would also add such factors as scarcity of ac- 
tivities or funds to cover SIPP, from the early stages all 
the way up to the final steps of innovation. 

In our study, the importance attributed to SIPP was 
revealed through an analysis of how present it was in 
the structure of the funding agencies. Some of the agen- 
cies defined knowledge translation in their mission state- 
ment and corporate objectives, while others did not. 
Some agencies created a specific branch, directorate or 
department to address SIPP. 
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The funds available to cover SIPP were embedded in 
push, pull, and linkage and exchange models. Few of the 
funding agencies that we studied offered funding based on 
all three of these models. Comparing, between these three 
models, how the results of a funded project on a particular 
topic is translated into knowledge is worth investigating 
further. Indeed, using a combination of all three models 
may lead to more significant research utilization than does 
favoring one model over another. 

While funding agencies emphasized the importance of 
SIPP, much work still needs to be done to ensure cover- 
age of all models of the knowledge transfer process and 
of all steps in the SIPP process. This lack is seen in our 
analysis of the programs being funded and in their in- 
ternal activities. The benefits obtained from funding 
SIPP and from SIPP activities are difficult to measure 
and compare because data reporting and collection is 
generally not systematic, as others have also noted [5]. 
Practice integration is largely predominant over policy 
integration, which is in line with Kastrinos' point that 
there is 'a marked trend away from mission-oriented 
policies and towards diffusion-oriented policies' [19]. 
Funding agencies are becoming more responsive to sci- 
ence than they are to policy [34]. 

Study limitations 

While some of the funding agencies studied showed a 
broad range of internal activities aimed at promoting 
SIPP, the published documentation of internal activities, 
either through websites or official documentation, was 
not consistent across data sources. "What is needed is 
more comparable data, and the integration of unpub- 
lished internal SIPP activities in development. This 
paper certainly does not provide an exhaustive list of the 
resources or mechanisms available for SIPP. Internal ac- 
tivities created within the agencies, or across them, were 
not always documented in reports and webpages access- 
ible to the public, while pilot activities under develop- 
ment were not yet available. Nonetheless, we are 
confident that we have covered the sections considered 
important by the funding agencies, as such sections are 
generally the most clearly identified sections. 

The empirical secondary data that we gathered from our 
sources are limited: they do not necessarily reflect the 
most recent and up-to-date decisions of funding agencies. 
We also need to consider that the benefits of research are 
mainly self reported by the researchers themselves, and 
that these benefits may be the result of either funding for 
the execution of research or funding aimed at encouraging 
the use of research by stakeholders. 

Furthermore, the relationship between the models 
(push, pull, and L&E models), or the different steps that 
are funded (synthesis, initiation, development, end of 



grant, science of SIPP, training in SIPP), and the actual 
results obtained have yet to be investigated. Additionally, 
the success stories purporting to demonstrate the actual 
integration of science into decision making seemed to 
depict the consequences of research results on practice, 
and less on service delivery or even policy. Thus, our 
analysis probably under represents the extent of internal 
activities related to SIPP and underestimates the results 
of funding. However, a quality verification across web- 
sites and some internal documents reporting on the ac- 
tivities of two Canadian institutions, Health Canada and 
CIHR, yielded the same general results. 

Contributions 

This paper makes an important contribution to research 
on SIPP because it goes beyond the two community 
model, which is based on the conceptualization that re- 
searchers produce knowledge and end users utilize 
knowledge, to include a variety of actors other than just 
producers and users of science. Other authors have 
pointed to the importance of having researchers frame 
the policy program [7], and to the major roles played by 
funding, research and policy actors. We focus on such 
issues in a detailed way, examining how science is inte- 
grated into policy and practice by an important actor in 
this process — funding agencies. 

One needs to keep in mind that while in this paper we 
have conceptualized producers as researchers from the 
scientific community, it is not always easy to make a dis- 
tinction between producers and users of science. For ex- 
ample, some producers of scientific knowledge work 
inside national departments and agencies. Similarly, 
users of knowledge for decision making can be found in 
a host of different entities: health organizations like hos- 
pitals; civil servants assigned to decision making posi- 
tions within funding agencies, civil servants outside the 
Ministry of Health, such as the Ministry of the Environ- 
ment; politicians deciding on health-related legislation; 
even university researchers whose work builds on previ- 
ous research. This heterogeneous sample of users may 
require a host of user-specific mechanisms for the initi- 
ation, development and dissemination of research. 

This paper sets the foundation for further discussion 
on SIPP in funding agencies in the healthcare domain. 
We have detailed the complex relationships that exist 
between knowledge use in the process of decision mak- 
ing at both the policy and practice levels. The other 
element— in essence, the flip side— worth investigating is 
knowledge development. This aspect, the integration of 
policy and practice into science, may be essential if we 
are to achieve a meaningful dialogue and reciprocal in- 
fluence between research and policy, and between re- 
search and practice [7]. Indeed, the production of new 
and useable knowledge to improve policy and practice 
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might find interesting lessons in the study of how policy 
and practice integrate into science. Insights for the inte- 
gration of policy into science can be learned from previ- 
ous studies on the Rothschild initiative in England 
[16,17]. Researchers conducted a seven-year formative 
evaluation of follow-up data on this attempt by the 
English Department of Health in the 1970s to influence 
and set the agenda for healthcare research. Recent edito- 
rials on collaborative research have highlighted the im- 
portance of maintaining an active relationship between 
research and the policy and practice side [11]. 

Case studies are very useful for identifying best practices 
in evidence-informed decision making. Of course, the in- 
tegration of knowledge/science into practice and policy 
does not depend on funding agencies alone [35]. Other or- 
ganizations like, for example, Canada's Policy Research In- 
stitute, inform policy, as do government entities like 
deputy ministers. In countries like Germany, there are 
mechanisms for such integration on advisory boards and 
government task forces [5], Future research could focus 
on whether establishing specific priority areas for funding 
is more effective in informing decision making than is 
funding based on pull, push, or L&E models. 
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